Capturing satisfactory photos under low light conditions using a hand-held camera can be a frustrating experience. Often the photos taken are blurred due to camera shake, or noisy. The brightness of the image can be increased in three ways. First, reducing the shutter speed can improve the brightness. But with a shutter speed below a safe level (the reciprocal of the focal length of the lens, in second units), camera shake will result in a blurred image. Second, using a large aperture can improve the brightness. A large aperture will, however, reduce the depth of field. Moreover, the range of apertures in a consumer-level camera is very limited. Third, setting a high ISO, can increase brightness. In traditional film photography ISO (or ASA) is an indication of how sensitive a film is to light. In digital photography, ISO measures the sensitivity of the image sensor. The lower the number, the less sensitive the camera is to light and the finer the grain. Higher ISO settings are generally used in darker conditions to obtain faster shutter speeds, however the tradeoff is noisier images. Thus, a high ISO image can be very noisy because the noise is amplified as the camera's gain increases. To take a sharp image in an environment of dim lighting, the best settings are safe shutter speed, large aperture, and high ISO. Even with this combination, the captured image may still be dark and very noisy. Another solution is to use flash, which unfortunately may introduce artifacts such as specularities and shadows. Moreover, flash may not be effective for distant objects. One approach is to reduce shutter speed to improve brightness and then try to remove the resultant blurriness.
In single image deblurring, the deblurring can be categorized into two types: blind deconvolution and non-blind deconvolution. The former is more difficult to achieve since the blur kernel is unknown. The real kernel caused by camera shake is complex, beyond the simple parametric form (e.g., single one-direction motion or a Gaussian) assumed in conventional approaches. Natural image statistics together with a sophisticated variational Bayes inference algorithm have been used to estimate the kernel. The image is reconstructed using a standard non-blind deconvolution algorithm. Pleasing results are obtained when the kernel is small (e.g., 30×30 pixels or fewer), but kernel estimation for a large blur is inaccurate and unreliable using a single image.
Even with a known kernel, non-blind deconvolution is still under-constrained. Reconstruction artifacts, e.g., “ringing” effects or color speckles, are inevitable because of high frequency loss in the blurred image. The errors due to sensor noise and quantizations of the image/kernel are also amplified in the deconvolution process. For example, more iterations in the Richardson-Lucy (RL) algorithm will result in more “ringing” artifacts.
Spatially variant kernel estimation has also been proposed. The image is segmented into several layers with different kernels. The kernel in each layer is uni-directional and the layer motion velocity is constant. Hardware-based solutions to reduce image blur include lens stabilization and sensor stabilization. Both techniques physically move an element of the lens, or the sensor, to counterbalance the camera shake. Typically, the captured image can be as sharp as if it were taken with a shutter speed 2-3 stops faster.
Single image denoising is a classic problem that is extensively studied. The challenge in image denoising is how to compromise between the tradeoff of removing noise while preserving edge or texture. Commercial softwares often use wavelet-based approaches. Bilateral filtering is also a simple and effective method widely used in computer graphics. Other approaches include anisotropic diffusion, PDE-based methods, fields of experts, and nonlocal methods.
One conventional technique consists of a primary sensor (high spatial resolution) and a secondary sensor (high temporal resolution). The secondary sensor captures a number of low resolution, sharp images for kernel estimation. However, this approach requires multiple images and special hardware. Another conventional technique uses a pair of images, where the colors of the blurred image are transferred into the noisy image without kernel estimation. However, this approach is limited to the case that the noisy image has a high SNR and fine details.